The present invention relates to a system for synthesizing a plurality of coded moving picture data as a multi-screen and a method thereof and, more particularly, a synthesizing method of getting moving picture data having a bit rate in response to the transmission rate when the synthesized moving picture data are transmitted via a predetermined transmission line.
In the system such as the TV conference system in which the moving pictures are transmitted, the network band required to transmit the moving pictures is assured by employing the dedicated line such as ISDN (Integrated Services Digital Network). In recent years, the transmission of the moving picture is requested in the environment such as the Internet whose network band is varied. In this case, various situations of the routes connecting the users reside in the Internet, and respective transmission bands are different. If the multi-screen moving pictures employed in the TV conference system are transmitted in real time in such environment, each user must transmits the images at a bit rate, that is smaller than the transmission band, toward the transmission line to which each user is connected. In the prior art, in order to prepare the moving picture data that is fitted to each transmission line, as shown in FIG. 12, there are the method (1) of generating the multi screen by copying the data that are received from the users having the small transmission band, and the method (2) of thinning the data to meet the moving picture of the lowest frame rate for the user having the narrow transmission band. Also, in order to generate the multi screen, there is employed the method of decompressing the moving picture data once and then coding the moving picture data again to meet the network band.
However, there are the problems such that (1) useless data are supplied to the network if the moving picture data are copied and employed, and (2) all images are lowered to the same frame rate by the thinning in the method, that thins the data in unit of frame, even if the video sent from the terminal to the server has the high frame rate. Also, the method (3) of generating the multi screen and transmitting it has the problem such that, since the approach of decompressing the moving picture data once and then coding the moving picture data again is employed to generate the multi screen, it is difficult to generate the multi screen in real time.
The present invention has been made to overcome the above problems, and it is an object of the present invention to enable to obtain moving picture data of a desired bit rate without re-coding when the multi-screen image is generated by synthesizing a plurality of moving picture data.
In order to achieve the above object, in the present invention, when the acquired original data are generated/transmitted as the multi-screen, bit rates of respective original data are adjusted such that they are suppressed within an upper limit of the transmission bit rate. As the adjusting method, a function of deciding the frame type at the time of synthesis is provided to vary the frame rate of the original data in synthesizing the multi-screen, and then a part of the frame is replaced with the frame type having the small data size.
Then, as the method of deciding the frame type, there is the first approach that provides the data X1 for managing temporarily the frame type in synthesis and the data Y1 for managing the number of times/frequency of the frame type (the type in which the size of the original data is assigned as it is, or conversely the type in which the size that is smaller than the size of the original data is assigned) assigned to the original data (video stream of the acquired source) in the synthesis, and then decides the data X1 in the descending order (ascending order) of the data Y1 not to exceed the transmission bit rate.
Accordingly, the frame type of a part of the moving picture in the multi-screen can be decided, and also the multi-screen moving picture can be generated at a desired bit rate in compliance with the data acquiring situation.
Also, there is the second approach that provides the data X2 for managing temporarily the frame type in synthesis, the data Z for managing the degree of importance (priority) among the original data, and the data Y2 for managing the number of times/frequency of the frame type (the type in which the size of the original data is assigned as it is, or conversely the type in which the size that is smaller than the size of the original data is assigned) assigned to the original data in the synthesis as the value calculated based on the value of the data Z, and then decides the data X2 in the descending order (ascending order) of the data Y2 not to exceed the transmission bit rate.
Accordingly, the multi-screen moving picture to which the frame rate decided based on the degree of importance is assigned can be generated at any bit rate.